Multimodal Referential Acts in a Dialogue Game: From Empirical Investigations to Algorithms
نویسندگان
چکیده
1 Abstract The paper presents an investigation into the question how a specification of the behaviour of Embodied Conversational Agents (ECAs) can be grounded in empirical investigations of human c onversational behaviour. We consider various empirical approaches and discuss one particular approach based on Dialogue Games in detail. We identify some pitfalls and problems which one faces when translating the results of such empirical investigations to algorithms for ECAs. Our discussion is illustrated by means of a specific investigation into the use of deictic referential acts in conversations.
منابع مشابه
Predicting Dialogue Acts for Intelligent Virtual Agents with Multimodal Student Interaction Data
Recent years have seen a growing interest in intelligent gamebased learning environments featuring virtual agents. A key challenge posed by incorporating virtual agents in game-based learning environments is dynamically determining the dialogue moves they should make in order to best support students’ problem solving. This paper presents a data-driven modeling approach that uses a Wizard-of-Oz ...
متن کاملFuzzy particle swarm optimization with nearest-better neighborhood for multimodal optimization
In the last decades, many efforts have been made to solve multimodal optimization problems using Particle Swarm Optimization (PSO). To produce good results, these PSO algorithms need to specify some niching parameters to define the local neighborhood. In this paper, our motivation is to propose the novel neighborhood structures that remove undesirable niching parameters without sacrificing perf...
متن کاملMultimodality and Dialogue Act Classification in the RoboHelper Project
We describe the annotation of a multimodal corpus that includes pointing gestures and haptic actions (force exchanges). Haptic actions are rarely analyzed as fullfledged components of dialogue, but our data shows haptic actions are used to advance the state of the interaction. We report our experiments on recognizing Dialogue Acts in both offline and online modes. Our results show that multimod...
متن کاملAn analysis of prosodic information for the recognition of dialogue acts in a multimodal corpus in Mexican Spanish
This paper presents empirical results of an analysis on the role of prosody in the recognition of dialogue acts and utterance mood in a practical dialogue corpus in Mexican Spanish. The work is configured as a series of machine-learning experimental conditions in which models are created by using intonational and other data as predictors and dialogue act tagging data as targets. We show that ut...
متن کاملAdvances in Natural Multimodal Dialogue Systems Advances in Natural Multimodal Dialogue Systems
We describe two major dialogue system segments: the first is an analysis module that learns to assign dialogue acts from corpora, but on the basis of limited quantities of data, and up to what seems to be some kind of limit on this task, a fact we also discuss. Secondly, we describe a Dialogue Manager which uses a representation of stereotypical dialogue patterns that we call Dialogue Action Fr...
متن کامل